---
title: Redundant routes (2nd internet connection) via A&A/AAISP L2TP
x-toc-enable: true
...

Introduction
============

This is a continuation of the main L2TP tunnel guide,
titled [L2TP tunnel router via Andrews and Arnold Ltd
(AAISP)](debian-l2tp-aaisp.md). If you've not yet read *that* guide, let alone
configured your router based upon it, you should read that guide first.

This tutorial will teach you how to add a *2nd* internet connection to your
L2TP tunnel router. For example, you might have docsis and vdsl (one as backup,
and one as main), or perhaps fttp and docsis (fttp as main, docsis as backup).

For the most part, the setup described on this page works very well. There are
a few minor issues, which this page will cover in detail.

The obvious benefit of this setup is reduced downtime. There is a slight delay
when auto-switching from main WAN to backup WAN, and vice versa, which I've
measured to be about 30-60 seconds in my tests, though your mileage may vary.

To be clear: *only one* connection will be used, *not* both, at any one given
time. *Bonding* will not be configured, in this tutorial.

How this works
==============

Introduction
------------

In the *main* L2TP guide, you are instructed to set a direct static route to
A&A's L2TP tunnel server, on your NIC connected to the main ISP router. In that
guide, you only have *one* internet connection, and *one* static route for the
L2TP tunnel through that internet connection, with **2 NICs** on the host (one
WAN and one LAN).

With this tutorial, you will alter that to create the following configuration:

* **3 NICs** in the host: one for main ISP, one for backup ISP, and one for
  LAN. In other words, two WANs and one LAN.
* Static route to *google* DNS server `8.8.8.8` on main WAN
* Static route to *google* DNS server `8.8.4.4` on backup WAN
* When L2TP is down, only *google DNS* is accessible. Not even A&A L2TP is
  directly accessible! It is possible to tell whether one or both WANs are
  online, by pinging `8.8.8.8` and `8.8.4.4`. This is a bit hacky, but it
  works. This guide will show you how.
* A script will constantly check which WAN is up, by pinging both of Google's
  DNS servers. Main is preferred; if that is up, a *static route* to A&A's L2TP
  server is routed through there.
* If the main WAN is down (based on ping), backup WAN is checked. If that is
  up, a *static route* to A&A's L2TP is set through that instead.
* If the backup internet connection is in use, it is assumed that the backup
  is reliable, and that main is unreliable for at least 15 minutes; therefore,
  the backup is used for 15 minutes. After 15 minutes, if main WAN is up, L2TP
  is re-routed back through main. If, after those 15 minutes, the main WAN is
  still down, the backup WAN shall continue to be used for another 15 minutes,
  repeating the process until main WAN is back online.

The script that reroutes L2TP traffic will run continuously, kicking in
automatically so that the switch-over happens without intervention.

A separate script will also write to the L2TP control file continuously,
attempting connection in a 5-second-delayed infinite loop. When the tunnel is
up, this loop does nothing and it is benign; when the tunnel is down, this
loop shall bring the tunnel back up. This mitigates any flaky connectivity in
the backup connection, when the backup WAN is currently in use during the 15
minute period. When main WAN is up, the main L2TP reroute script also handles
the L2TP control file, but such handling is (due to design) not performed at
all while the backup internet connection is in use.

Caveat: this means that if the *backup* is down, but the main comes back online
while the backup is down, you will have at maximum about 15 minutes of extra
downtime; this is a known design flaw in the configuration, but this is an
absolute worst case scenario. It's a trade-off that makes the script much
simpler, but patches are welcome if you wish to improve the setup described in
this guide.

Caveat: Google DNS for clients
------------------------------

One caveat is that clients behind the L2TP tunnel router will, by default,
not be able to use Google DNS on IPv4. The IPv6 Google DNS addresses will still
work for clients.

This *can* optionally be fixed, and such will be covered later in the guide.

Interface names
---------------

Later in this guide, you will see interface names used for networking. They
are:

* `eth0`: main internet (WAN), via NAT `192.168.0.1/24` subnet.
* `eth1`: backup internet (WAN), via NAT `192.168.1.1/24` subnet.
* `eth2`: LAN, routed via L2TP over `eth0` or `eth1`, for routed /28 IPv4
  and /64 IPv6 subnets (**your subnet sizes may differ**).

Configuration
=============

I like to thoroughly explain everything, before actually getting down to
business. I hope the above explanations made sense.

The following configurations shall implement what has been described. Compare
these configurations to your existing setup, based on the original L2TP guide,
and adapt accordingly.

**You MUST adapt the interface names and IP addresses to your setup.**

/etc/network/interfaces
-----------------------

It is here, where we set the static routes to Google DNS, for ping tests.

Unlike the `interfaces` file described in the main L2TP guide, static routes
to A&A's L2TP server are *not* directly set in this file.

Configured as follows:

```
# The loopback network interface
auto lo
iface lo inet loopback

# main internet (WAN)
auto eth0
allow-hotplug eth0
iface eth0 inet static
       address 192.168.0.101
       netmask 255.255.255.0
       network 192.168.0.0
       broadcast 192.168.0.255
       up /sbin/ip route add 8.8.8.8/32 via 192.168.0.1 dev eth0

# backup internet (WAN)
auto eth1
allow-hotplug eth1
iface eth1 inet static
       address 192.168.1.101
       netmask 255.255.255.0
       network 192.168.1.0
       broadcast 192.168.1.255
       up /sbin/ip route add 8.8.4.4/32 via 192.168.1.1 dev eth1

# LAN (consisting of one public /28 IPv4 and one public /64 IPv6 subnet)
# A&A might only give you a /29; they only advertise IPv4 subnets at all,
# for the *business class* L2TP service, not domestic/light L2TP tier
# IPv4 /28:
auto eth2
allow-hotplug eth2
iface eth2 inet static 
   address 81.187.172.129
   netmask 255.255.255.240 
   dns-nameservers 217.169.20.20 217.169.20.21
# IPv6 /64
iface eth2 inet6 static 
   address  2001:8b0:b95:1bb5::1
   netmask 64 
   dns-nameservers 2001:8b0::2020 2001:8b0::2021
```

Crontab
-------

In the main L2TP guide, we had you add this rule to root's crontab:

	* * * * * echo "c aaisp" > /var/run/xl2tpd/l2tp-control

You *can* leave this rule in place, but we will not rely upon it anymore. You
may aswell delete this. Do so by running the following command as root:

	crontab -e

Remove the line, save the file, and exit.

/sbin/reroute
-------------

Create this file as root, inserting the following contents:

```
#!/bin/bash

# 8.8.8.8 is routed through main internet (192.168.0.1)
# 8.8.4.4 is routed through backup internet (192.168.1.1)

# who really cares about google dns anyway?

l2tp_on_main="false"
l2tp_on_backup="false"

while true
do

main_up="false"
backup_up="false"

ping -c 1 8.8.8.8 && main_up="true"
if [ "${main_up}" = "false" ]
then
        ping -c 1 8.8.4.4 && backup_up="true"
fi

if [ "${main_up}" = "true" ]
then
        if [ "${l2tp_on_backup}" = "true" ]
        then
                l2tp_on_backup="false"
                echo "d aaisp" > /var/run/xl2tpd/l2tp-control
                sleep 15
        fi
        /sbin/ip route del 90.155.53.19/32 via 192.168.1.1 dev eth1
        /sbin/ip route add 90.155.53.19/32 via 192.168.0.1 dev eth0
        echo "c aaisp" > /var/run/xl2tpd/l2tp-control
        sleep 60
        ping -c 1 1.1.1.1 && l2tp_on_main="true"

elif [ "${backup_up}" = "true" ]; then
        if [ "${l2tp_on_main}" = "true" ]
        then
                l2tp_on_main="false"
                echo "d aaisp" > /var/run/xl2tpd/l2tp-control
                sleep 15
        fi
        /sbin/ip route del 90.155.53.19/32 via 192.168.0.1 dev eth0
        /sbin/ip route add 90.155.53.19/32 via 192.168.1.1 dev eth1
        echo "c aaisp" > /var/run/xl2tpd/l2tp-control
        sleep 60
        ping -c 1 1.1.1.1 && l2tp_on_backup="true"
        if [ "${l2tp_on_backup}" = true ]
        then
                # assume backup is reliable, and that main is up/down
                # constantly. we should stay on the backup for at least
                # 15 minutes, to ensure stability
                sleep 900
        fi
else
        # both internetz are down. try again in 15 seconds
        sleep 15
fi

done
```

MAKE SURE to set this file executable; as root, do:

	chmod +x /sbin/reroute

/sbin/autoconnect
-----------------

You may have already created this file, and set it to run automatically,
when you followed the main L2TP. If this is so, then you can ignore the
instruction below.

Create this file as root, inserting the following contents:

```
#!/bin/bash

while true
do
        sleep 5
        echo "c aaisp" > /var/run/xl2tpd/l2tp-control
done
```

MAKE SURE to set this file executable; as root, do:

	chmod +x /sbin/autoconnect

That is all. There is not much changed, relative to the main L2TP guide.

Auto-start
==========

Network interfaces not starting
-------------------------------

In my test setup, I was using 3 USB NICs, and sometimes at system startup,
they would not show up on the interface list (seen by running `ip a`) in
Debian.

My workaround is indeed quite gruesome and low-tech: I simply unplugged them,
plugged them back in, checked again to make sure they all appeared in `ip a`,
and then I did this as root:

	systemctl restart networking

Your NICs might be a bit more robust than what I was using.

Avoid crontab
-------------

The main L2TP guide simply had you write a crontab rule. In this guide, so far
you've already been instructed to delete the crontab rule pertaining to
xl2tpd's control file.

In my testing, use of crontab for this purpose led to whacky results.
Namely, each script running *twice*. However, on crontab, you could otherwise
add a line like this:

	@reboot /sbin/reroute
	@reboot /sbin/autoconnect

Use of a *crontab* is ill advised.

Or run manually!
----------------

Basically, you just need to run these two commands:

	/sbin/reroute
	/sbin/autoconnect

However, it is important that these scripts aren't *killed*. It is therefore
wise to consider one of the following:

* Log in as root, and install tmux or GNU Screen. Run it in a tmux or screen
  session, indefinitely. This means that you have to manually
* Have these two script run automatically, at boot time

In my specific situation, the router is battery-backed and encrypted, so I
have to handle encryption at boot time anyway, thus making the question
of *autostart* irrelevant. I simply run these commands manually, when I boot
the router, and then I simply leave them to run indefinitely.

/etc/rc.local in Debian 11
--------------------------

I'm a BSD user at heart, so I'm quite averse to creating systemd service files.
This and other guides that Fedfree is starting out with, they target Debian,
but it is my intention to cover the BSDs at some point. Linux, specifically
Debian, is the most popular choice among normal people, so I'm heavily focusing
on that first, but BSD lets you do all sorts of crazy whacky things that would
be next to impossible (or otherwise unreliable) in Linux.

Systemd is a scourge on all of Unix.

Debian 11 has done away with `rc.local`, at least in the default configuration.
You may still *re-enable* it, like so:

Create the file `/etc/rc.local` with these contents:

```
#!/bin/sh

/sbin/reroute &
/sbin/autoconnect &

exit 0
```

NOTE: the `-e` directive is *NOT* set here, because errors are expected when
dealing with routing-related scripts. Otherwise, it would be good practise to
put this at the start of the script:

```
#!/bin/sh -e
```

Mark the file as executable:

	chmod +x /etc/rc.local

You will also create the file `/etc/systemd/system/rc-local.service` and
insert these contents:

```
[Unit]
Description=/etc/rc.local
ConditionPathExists=/etc/rc.local
 
[Service]
Type=forking
ExecStart=/etc/rc.local start
TimeoutSec=0
StandardOutput=tty
RemainAfterExit=yes
SysVStartPriority=99
 
[Install]
WantedBy=multi-user.target
EOF
```

Now start it, with these commands:

	systemctl daemon-reload
	systemctl start rc-local

To enable it at every boot:

	systemctl enable rc-local

You might check the status, for errors:

	systemctl status rc-local

The `rc.local` file is executed after everything else is, at startup, just
before TTY/X login.

Testing
=======

It is strongly recommended that you do *not* use SSH for this. You should log
in directly, at the tty prompt of your router.

While logged in to the L2TP tunnel router, *directly*, do this:

	watch -n .2 ip route

If the main internet connection is in use, you will see a line looking similar
to this:

```
90.155.53.19 via 192.168.0.1 dev eth0
```

If the backup internet connection is in use, you will see a line looking like
this:

```
90.155.53.19 via 192.168.1.1 dev eth1
```

If the tunnel connection is up, you will see something like this (and you will
obviously have use of the internet):

```
81.187.81.187 dev ppp-aaisp-l2tp proto kernel scope link src 81.187.232.11
```

If the `reroute` and `autoconnect` scripts are running, and both internet
connections work, you have the ideal test conditions. You may now try the
following:

* If traffic is currently routed through main, simply test the internet. Just
  do whatever you wish to do on the internet.
* Second, physically disconnect (on the WAN router side, not L2TP side) the
  cable, but not in such a way that there is no electrical connection between
  L2TP router's main WAN port; use of a switch in-between is assumed, where
  you also have the backup WAN router connected.
* With the cable removed, wait and see if `ip route` output changes. It may
  take a few moments, so be patient.
* When you see the IP route change to the backup WAN IP, that doesn't mean
  the L2TP is up yet; check also that the route via `ppp-aaisp-l2tp` exists
  and that the internet works. You might have to wait a few moments, even a
  minute or so.
* If you've confirmed that the internet now works on backup, with main
  unplugged, plug the main back in.
* Wait about 20 minutes, and test the internet again to see if it's back on
  main. You will do this, *after* confirming that the output of `ip route` has
  changed accordingly.

If it works, then you deserve a medal for putting up with my writing. Well
done!

This is a *solid* configuration, that can be used for all sorts of redundant
server hosting, or even just regular internet at your office, for normal
internet usage in situations where you *must* be always online.

You are now part of the 0.1% *of* the 0.1% of nerds on the internet who host
their own servers, that also have redundant internet. Self-hosted, redundant,
dual stack IPv4 and IPv6 servers in one's living room is indeed rare. You were
wily enough to get this far.

Security
========

If you're running L2TP behind a NAT router provided by your main ISP, you
might consider *double* NAT, with a NAT router in front of it running something
more secure like OpenWRT/pfSense; alternatively, you may be able to use your
ISP's router/modem in bridge (modem-only) mode; if the ISP provided a modem and
router as two separate devices (rare these days), you could replace the router
entirely.

It is important that your ISP's router cannot, itself, set static IPs on the
LAN as routed from your L2TP tunnel. You *must* assume that your ISP-assigned
router is malicious, in line with the the security discipline known as
the *principle of least privilege*.

This is up to you, as the specifics will vary according to your network and
wiring. It is beyond the scope of this article, but you should take security
seriously on your network.

As stated in the main L2TP article, and unchanged by the configuration in this
one, you are running an open network. Any hosts behind the L2TP tunnel will
have a *direct connection to the internet, with all ports open, directly
pingable. It is imperative that you consider packet filtering on each host
plugged in to your network* - oldschool firewall boxes by themselves are best.
These are *oldschool* because it's common nowadays for routers to also do
firewalling, but in *this* setup, it's much more feasible to have the routing
and firewalling actually be handled by separate boxes (my preferred setup).

If feasible, make sure SSH is only listening on a local address. Just add
a `10.0.0.0/8` subnet IP or something, on the LAN side of your L2TP router,
then set that statically on a client when you need to use SSH for managing
the router. This can be set via the `ListenAddress` directive in `sshd_config`.
Disabling password authentication (in favour of key-only authentication) is
also recommended, when dealing with SSH, that and things like `fail2ban` - all
of this could be covered in a future tutorial.

Re-enable Google DNS (clients)
====================

*As of 2 January 2023, these instructions are untested.*

Just think about this for a moment:

```
ping -c 1 8.8.8.8 && main_up="true"
if [ "${main_up}" = "false" ]
then
        ping -c 1 8.8.4.4 && backup_up="true"
fi
```

The above logic exists in your `reroute` script, on the assumption that
one google DNS IP is routed through first WAN, and other DNS IP through second
WAN IP.

If those static routes were to be *deleted*, after L2TP goes online, then both
Google DNS IPs would work perfectly, for clients behind the L2TP router. This
deletion would occur in the if-up script, which you will have created when
implementing the instructions from the main L2TP guide.

That if-up script *only* executes when L2TP goes up.

The if-down script could be used to *re-add* those static routes.

In other words, when L2TP is offline we allow google DNS routing via WAN IPs
outside of L2TP (which does not exist), for the purposes of our ping test. We
then allow Google DNS to route via L2TP tunnel when the tunnel is online.

If the L2TP tunnel is online, your `reroute` script will ping `8.8.8.8` and
conclude that "main" is up, regardless of whether it's main or backup, and
it will just route accordingly.

It *should* work. Anyway, you could try the following modifications:

/sbin/reroute
-------------

Look at these lines:

```
main_up="false"
backup_up="false"

ping -c 1 8.8.8.8 && main_up="true"
if [ "${main_up}" = "false" ]
then
        ping -c 1 8.8.4.4 && backup_up="true"
fi
```

Below the line where it says `backup_up="false"`, but before the line that
says `ping -c 1 8.8.8.8 && main_up="true"`, add:

```
l2tp_up="false"
ping -c 1 1.1.1.1 && l2tp_up="true"
if [ "$l2tp_up" = "true" ]
then
	if [ "${l2tp_on_backup}" = "true" ]
	then
		sleep 900
	fi
	continue
fi
```

/etc/ppp/ipv6-up.d/0000-defaultroute
------------------------------------

It is assumed that you have these contents:

```
#!/bin/bash
/bin/logger $1 is up
if [ $1 = "ppp-aaisp-l2tp" ]; then
    /bin/logger "AAISP over L2TP circuit is online; adding routes"
    /sbin/ip route add default dev ppp-aaisp-l2tp scope link
    /sbin/ip -6 route add default dev ppp-aaisp-l2tp scope link
fi
```

You will change it to say this:

```
#!/bin/bash
/bin/logger $1 is up
if [ $1 = "ppp-aaisp-l2tp" ]; then
    /bin/logger "AAISP over L2TP circuit is online; adding routes"
    /sbin/ip route add default dev ppp-aaisp-l2tp scope link
    /sbin/ip -6 route add default dev ppp-aaisp-l2tp scope link
    /sbin/ip route del 8.8.8.8/32 via 192.168.0.1 dev eth0
    /sbin/ip route del 8.8.4.4/32 via 192.168.1.1 dev eth1
fi
```

/etc/ppp/ipv6-down.d/0000-defaultroute
--------------------------------------

It is assumed that you have these contents:

```
#!/bin/bash
/bin/logger $1 is down
if [ $1 = "ppp-aaisp-l2tp" ]; then
    /bin/logger "AAISP over L2TP circuit is offline; removing routes"
    /sbin/ip route del default dev ppp-aaisp-l2tp scope link
    /sbin/ip -6 route del default dev ppp-aaisp-l2tp scope link
fi
```

You will change it to say this:

```
#!/bin/bash
/bin/logger $1 is down
if [ $1 = "ppp-aaisp-l2tp" ]; then
    /bin/logger "AAISP over L2TP circuit is offline; removing routes"
    /sbin/ip route del default dev ppp-aaisp-l2tp scope link
    /sbin/ip -6 route del default dev ppp-aaisp-l2tp scope link
    /sbin/ip route add 8.8.8.8/32 via 192.168.0.1 dev eth0
    /sbin/ip route add 8.8.4.4/32 via 192.168.1.1 dev eth1
fi
```

That is all.
